In the processing of various types of signals and images, including one dimensional signals (e.g., a function only of time) or multidimensional signals (e.g., an image signal which is a function of two or three dimensional space), it is often desirable to improve the quality of the signal from the signal acquisition system to correct for distortions introduced by the acquisition system. Such processing must be carried out in the inevitable presence of noise. Assuming a linear acquisition system, the output signal g from an acquisition system having an impulse response function h, responding to an actual signal f, and contaminated by random noise n, can be expressed as: EQU g=hf+n
where represents the convolution operation.
For many types of signal acquisition systems of great practical importance, the impulse response h is bandlimited in the frequency domain and varies over different portions of the signal, and cannot be calibrated accurately by empirical methods. Such acquisition systems include, by way of example, various types of microscopes, including widefield and confocal microscopes, scanning electron microscopes, scanning transmission electron microscopes, many other types of spectroscopic instruments including X-ray, infrared, etc., and one and two-dimensional gel electrophoresis processes.
One area of particular interest is fluorescence microscopy, which permits the imaging of very weak light signals from luminescent molecules. In fluorescence microscopy, the detection optics of the microscope system is modified so that only certain wavelengths of light from excited molecules reach the detector, and the use of such microscopy with various fluorescent probes makes the technique very useful for imaging cells or parts of cells, indicating concentration of ions, in the staining of specific organelles, in detecting and quantifying membrane potentials, and in the detection of target molecules both inside and outside of the cell.
In conventional fluorescence microscopy, the fluorescent intensity at a given point in a stained region can provide quantitative information about the target molecules. However, it is well known that in any microscope when a point object is imaged in three dimensions, the point is imaged not only in the plane of focus but also in the planes around the focal plane. Furthermore, the intensity of the point falls off more slowly in the direction of the optical axis as compared to a direction perpendicular to the optical axis. Therefore, in a 3-D specimen being imaged, the intensity measured at any point in a given plane will be influenced by contributions of the intensities of neighboring points, and the Contributions of points from out-of-focus planes have a greater effect on the quality of the image than contributions from in-focus points. In the spatial frequency domain, these distortions are manifested in two different ways. First, outside of a biconic region of frequencies in the three dimensional spatial frequency spectrum, the spectrum of the image is degraded by a strong low-pass function, and secondly, inside the biconic region of frequencies, all the frequency components of the image are removed during the image formation process. The severity of these distortions depends on the numerical aperture of the objective lens used for obtaining 3-D images. Higher numerical aperture lenses yield better 3-D images than lower numerical aperture lenses because of their superior optical sectioning capability. However, in addition to such optics-dependent distortions, image data is invariably corrupted by imaging noise. Thus, it is a significant problem to obtain good qualitative and quantitative information about the actual specimen from degraded images because if out of focus information is not excluded, unpredictable errors can affect both the visual and quantitative analysis of the specimen.
A relatively recent development in microscopy is the confocal scanning microscope, in which only a single diffraction-limited spot in the specimen is illuminated at a time, with the light emitted from the illuminated spot being focused through an aperture which is "confocal" with the spot of illumination. An image of a three-dimensional specimen is acquired by scanning a frame in the in-focus image plane and then moving the image plane to acquire additional frames of data until the entire specimen is scanned. In theory, in a confocal microscope the out-of-focus information can be removed, but in practice reducing the size of the confocal aperture results in a corresponding reduction in the signal-to-noise ratio.
Thus, in both conventional and confocal microscopy, as well as in other signal processing systems, it is desirable to extract information from the original signal which would otherwise yield, when the signal is displayed as an image, a blurred, noisy image. The process of extracting unblurred images is called deconvolution. Typical prior deconvolution processes require knowledge of the blurring function (impulse response) of the signal acquisition system (e.g., a conventional microscope or a confocal microscope) called the point spread function (PSF) o A technique utilizing projection onto convex sets and a form of Landweber iteration has been used for image restoration where the exact PSF is available through empirical measurements, e.g., by measuring the PSF of a microscope which involved imaging a diffraction limited point source. See Gopal B. Avinash, Computational Optical Sectioning Microscopy with Convex Projection Theory with Applications, Ph.D. thesis, 1992, University of Michigan, University Microfilms International, Inc., and Gopal B. Avinash, Wayne S Quirk, and Alfred L Nuttal,"Three-Dimensional Analysis of Contrast Filled Microvessel Diameters," Microvascular Research, Vol 45, 1993, pp 180-192.
In most practical situations a complete knowledge of the PSF is not available, or the calibration procedures have to be carried out before each new specimen is imaged, which is often so time consuming as to make the procedure impractical. Thus, attempts have been made to develop procedures called blind deconvolution which do not require complete prior knowledge of the PSF, and which attempt to simultaneously identify both the blurring function (the PSF) and the ideal unblurred image using the observed image data which are blurred and noisy.
A special type of image formation occurs in microscopy where the image of a specimen is degraded by a band-limited PSF In the paper by T. J. Holmes, "Blind Deconvolution of Quantum Limited Incoherent Imagery: Maximum Likelihood Approach," J. Opt Soc. Am. A, Vol 9, 1992, pp. 1052, et seq., a method based on maximum likelihood estimation for blind deconvolution was described. The success of the method was attributed in the paper to properly constraining the PSF estimate and the specimen estimate in the simulation studies. In the paper by V. Krishnamurthi, et al., "Blind Deconvolution of 2-D and 3-D Fluorescent Micrographs," Biomedical Image Processing and Three-Dimensional Microscopy, Proc. SPIE 1660, 1992, pp. 95-104, blind deconvolution was applied to deblur three dimensional fluorescent micrographs from real specimens. A further form of this procedure was described in a paper by T. J. Holmes, et al., "Deconvolution of 3-D Wide Field and Confocal Fluorescence Microscopy Without Knowing the Point Spread Function," Proceedings of Scanning '93, 1993, III-57-59, where the authors outlined the specific constraints on the PSF, on the basis that an empirical band-limit protocol has provided comparable results to deconvolution with the known PSF. These initial attempts to estimate images of specimens in the absence of complete knowledge of the PSF have several disadvantages. The method requires hundreds of iterations to converge to the final solution and, therefore, requires significant additional computation accelerating hardware to perform even a very modest size (64.times.64.times.64) 3-D image in a reasonable amount of time. The procedure constrains the PSFs using theoretically obtained bandlimit parameters, which generally are not appropriate. See, D. A. Agard, et al., "Fluorescence Microscopy in Three Dimensions, " Methods in Cell Biology, Academic Press, Inc., Vol. 30, 1989, pp. 353-377.
Blind deconvolution is also of interest in other applications of signal processing generally and image signal processing in particular. The application of maximum likelihood estimators using expectation maximization schemes for simultaneous blur and image identification in two dimensions has been actively pursued. See, e.g., M. I. Sezan, et al., "A Survey of Recent Developments in Digital Image Restoration," Optical Engineering, Vol. 29, 1990, pp. 393-404. Generally, these procedures have been computationally very intensive and require the use of computers with array processors to complete the processing in a reasonable amount of time. Other methods which have been proposed have the same disadvantages. See, e.g., B. C. McCallum, "Blind Deconvolution by Simulated Annealing," Optics Communications, Vol. 75, No. 2, 1990, pp. 101-105, which describes the use of simulated annealing, and B. L. K. Davey, et al., "Blind Deconvolution of Noisy Complex Valued Image," Optics Communications, Vol 69, No. 5-6, 1989, pp. 353-356, which describes the use of Weiner filtering for simultaneous blur and image identification.